Using language as knowledge

With the spread of computers and the Internet, hearing news reports of topics such as people playing shogi or chess against computers has become commonplace. We live now in an era where computers appear on game shows. A research and development project at IBM is currently working on a computer system which will participate as a contestant on the popular American quiz show "Jeopardy!". It contains a question response system that combines information retrieval and language processing technologies, and research is underway on question answering technologies that can also handle so-called "trick questions". NTCIR is an internationally active, workshop style project whose objective is the advancement of these kinds of information access technologies. The expression "information access" was chosen because NTCIR's objective is a system for "supporting the users to create of new value from massive amounts of information"by retrieving relevant information for the users and supporting the users to utilize the information in the documents. Therefor NTCIR is researching information retrieval, technologies for supporting the users to util ize the information in documents such as question answering, summarization, opinion analysis , trend analysis , etc. and search appropriate questions. The project, started in 1997, subsumes several research divisions related to information access technologies, and each division is operated by a group of researchers, who function as "organizers". The importance the project places on the workshop-style approach can also be seen in research division selection. The project's management does not unilaterally establish research divisions. Instead, researchers in related fields collect research proposals, and decisions are made after a committee considers them from the perspectives of their content, feasibility, international research trends and technological trends, and social value. Each activity cycle lasts a year and a half, with NTCIR-8 (its 8th cycle) currently underway. A single cycle's process is as follows. First, organizers propose research division objectives and evaluation methodology, discussing them with researchers who wish to participate, and determining final evaluation methods and data. After this, organizers distribute shared document datasets and query datasets. Participants use these datasets in their search experiments, thereby performing verification of the systems they have developed. These results are collected, evaluated by human assessors, and correct answers are created. In some cases, correct answer candidates, created in advance, can be used, in which case many participants evaluate and verify them, increasing the reliability and validity of the correct answer proposals. Last, the verification and evaluation results are gathered in the form of research papers, which report the achievements of the research, bringing the cycle to a close. The document data, query data, and correct answers are referred to collectively as a "test collection". These are, of course, repeatedly used by researchers participating in NTCIR for further research, but they are also made publicly available to non-NTCIR participants, so that a wider research community can test their methodologies against a validated standard. This contributed to the efficient advancement of research activities. Verifying the effectiveness of information access technologies requires, for experimentation, a large number of users and questions. However, during initial stages of research, where verification is needed every time a new idea emerges, it is difficult to gather a large number of users and perform long-term testing. By using test collections, the effectiveness of research ideas can be verified immediately, and repeatedly, through research lab experimentation, rapidly accelerating research progress.